Fully Dynamic Inference with Deep Neural Networks

Modern deep neural networks are powerful and widely applicable models that extract task-relevant information through multi-level abstraction. Their cross-domain success, however, is often achieved at the expense of computational cost, high memory bandwidth, long inference latency, which prevents their deployment in resource-constrained time-sensitive scenarios, such as edge-side self-driving cars. While recently developed methods for creating efficient making real-world more feasible by reducing model size, they do not fully exploit input properties on a per-instance basis to maximize efficiency task accuracy. In particular, most existing typically use one-size-fits-all approach identically processes all inputs. Motivated fact different images require feature embeddings be accurately classified, we propose dynamic paradigm imparts convolutional with hierarchical dynamics level layers individual filters/channels. Two compact networks, called Layer-Net (L-Net) Channel-Net (C-Net), predict or filters/channels redundant therefore should skipped. L-Net C-Net also learn how scale retained computation outputs By integrating into joint design framework, LC-Net, consistently outperform state-of-the-art frameworks respect both classification On CIFAR-10 dataset, LC-Net results up $11.9\times$11.9× fewer floating-point operations (FLOPs) 3.3 percent higher accuracy compared other methods. ImageNet achieves notation="LaTeX">$1.4\times$1.4× FLOPs 4.6 Top-1 than